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Most states have put in place curriculum standards and state-developed assessments 
to monitor the implementation of those standards. Most state standards define expected 
outcomes, that is, what students need to know and be able to do, but do not mandate 
specific strategies or pedagogy used by local districts. As researchers Paul Black and 
Dylan William (1998) note, standards are raised only by changing what happens in the 
classroom, beginning with teachers and students. This Digest describes a program 
used by two educators to help teachers improve instruction through a deeper 
understanding of state standards and test specifications. 

DEVELOPING A KNOWLEDGE BASE 



Understanding how standards-based state tests are constructed is the first step in being 
able to use them to guide and improve instruction. A test is essentially a sample of 
questions or activities that reflect a large body of knowledge and mental processes 
associated with an academic subject area such as language arts, mathematics, 
science, etc. School districts are under pressure to perform well on state tests and often 
use a test preparation strategy of giving students sample tests from commercially 
prepared workbooks or state-released items to get ready for state tests. Although this 
strategy can be useful for providing general information regarding student strengths and 
weaknesses as related to the samples, it should not be the only method used by 
teachers because it does little to educate them about how to understand and use state 
tests, standards, and test specifications. This article recommends a three-part process 
of delineation, alignment, and calibration for developing an understanding of state 
assessments and using that understanding to improve instruction. The New Jersey 
Core Curriculum Content Standards are used to model the process, but it can be 
applied to any state's or district's standards. 

DELINEATION 



Delineation is the process of identifying all aspects or dimensions of a particular subject 
domain. It involves using state testing documents that describe each content area of the 
assessment, including test specifications, specific skill cluster information, subject area 
frameworks, assessment examples and exemplars, and the state standards. 

Delineation requires an examination of these documents for assessment dimensions 
such as content, cognitive level and complexity. A thorough delineation might also 
include analysis of additional factors such as the test format and difficulty level of the 
questions. 

The New Jersey Standards include macro, or big picture, statements and cumulative 
progress indicators that provide details about general performance expectations. The 
standards include knowledge specifications, which describe the specific processes and 
content that all students must know by the end of fourth grade (also known as content 
standards) as well as problem-solving specifications, which describe what students 
should be able to do with the content knowledge (also known as process standards). 
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The following example is excerpted from the 4th grade New Jersey mathematics 
standards and test specification manuals. 

Macro Standard 4.1 : All students will develop the ability to pose and solve mathematical 
problems in mathematics, other disciplines, and everyday experiences. 

Cumulative Progress Indicator 4.1 .2: Recognize, formulate, and solve problems arising 
from mathematical situations and everyday experiences. Test Specification Manual - 
Cluster IV Discrete Mathematics: Knowledge (content standards): Students should have 
a conceptual understanding of: Tree diagram 

Problem Solving (process standards): In problem solving settings, students should be 
able to: Draw and interpret networks and tree diagrams 

After reviewing the entire body of standards and test specifications for fourth-grade 
mathematics in New Jersey, a teacher would be able to identify seven distinct 
mathematics strands or dimensions: Numeration and Number Theory, Whole Number 
Operations, Fractions and Decimals, Measurement/Time/Money, Geometry, Probability/ 
Statistics, and Pre-algebra. The test specifications for that exam imply that the 
mathematics test questions are primarily composed of problem-solving tasks. 

Therefore, it is safe to assume that test questions will require thinking in the application, 
analysis, and perhaps synthesis and evaluation levels of cognition. 

Once teachers and administrators specify all of the subject area dimensions, the 
following activities can begin: 



* Selecting and designing classroom assessments and practice questions. 



* Revising and designing curriculum that is congruent with the content identified in the 
state standards and the district's delineation of the state-designed exams 



* Designing teacher training using instructional techniques that support these 
dimensions. 

ALIGNMENT 



During the alignment phase, administrators and teachers work to identify, analyze, 
generalize, and describe the links between the various elements associated with the 
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subject area previously delineated and the sample questions selected for practice or 
classroom activities to assess student progress. The sample questions and student 
assessments can be derived from several sources, including state-released test items, 
commercially manufactured test preparation materials, or teacher-made activities. 
Teachers and administrators examine linkages in the materials, organization, textbooks, 
instructional strategies and other elements described in the curricula and used in daily 
instructional activities to ensure consistency with the district's delineation of the state 
assessment. 

Using and understanding the test specifications become even more important at this 
stage. Returning to the example above, a teacher seeking to ensure that students would 
be able to understand tree diagrams and solve problems using them would complete 
several alignment tasks: 

1 . Review classroom resources, curriculum, textbooks, teacher activities, student 
thinking strategies, and tests to ensure that the test specifications and macro standards 
are addressed on the knowledge and problem solving level. Do the teacher resource 
materials and classroom instruction address the proper skills? 

2. Review the above factors to ensure congruency between the level o difficulty required 
by the standards and specifications, and the difficulty of the actual teacher resources 
and activities. Do the teacher's tests, lessons, and activities match the difficulty level 
required by the standards and specifications? 

3. Consider format. Although less important than skills and difficulty, the teacher 
resources, activities, and tests should familiarize the students with state test question 
formats. Teachers must align classroom assignments and activities to the subject area 
delineation to ensure congruency. 

CALIBRATION 



After completing the delineation and beginning the alignment processes, calibration 
begins. Calibration is the act of conducting communications and interactions with 
teaching staff based on the information identified in delineation and used in alignment. 
The calibration process ensures that the conceptualization of content, cognitive 
process, complexity, formats, etc., is consistently understood for each subject area. 
Calibration, in its simplest form, is designing classroom instruction, activities, and 
assessments that are congruent with content area delineation and alignment. Using the 
mathematics vignette as an example, one can begin to see how the process takes 
place. 

Figure 1 . Represents the sequence of events leading up to Calibration. 



Delineation 



Page 4 of 8 



ED458215 2001-05-00 State Standards, Assessments and Instruction. ERIC Digest. 



ERIC Resource Center 



WWW . eric . ed . gov 



1 . Teacher performs content area delineation and chooses a unit of focus. In this case 
Standard 4.1 , Cluster IV-Discreet Mathematics-tree diagrams 



Alignment 

2. Teacher examines a compares classroom resources, local curriculum, activities, skills 
level of difficulty, format, and tests to the Standards and test specifications to ensure 
congruency. 



Instruction 

3. Teacher uses and implements calibrated activities, tasks, resources and lessons. 



Calibration 

4. Teacher designs lessons and activities, gathers resources, and creates tests that are 
congruent with the skills and level of difficulty for the Standards, test specifications and 
curriculum. 

Figure 1 . Delineation, Alignment, and Calibration Flow of Events 

Matt has four channels on his television. He has channels 2,3,4 and 5. If Matt watches 
only two channels each night, haw many different combinations of channels can he 
watch? Show all your work, and explain your answer. Matt can watch (2,3), (2,4), (2,5) 
or (3,2), (3,4), (3,5) etc. 

A 4th grade teacher completing delineation and alignment and discovering that her/his 
program was missing a unit on discrete mathematics would develop objectives related 
to understanding, using, and interpreting tree diagrams. Figure 2 is a sample 
activity/test question created by 4th grade teacher Terry Maher to begin addressing the 
aspect of discrete math noted in the Cluster IV test specification. 

Calibration is any action that helps teachers design activities and construct 
assessments based on the dimensions of state assessments and standards. This 
process helps to foster a collective understanding and agreement of the dimensions and 
domains of each content area. It should be a team effort based on group inquiry. 

USING SCORE REPORTS TO IMPROVE 
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CALIBRATION 



As teachers gain a better understanding of how student work reflects the standards and 
test specifications through delineation, alignment, and calibration, their efficiency and 
accuracy at identifying which students are meeting the standards should increase. 
Herein lies the usefulness of score reports sorting students into categories of varying 
proficiency. A student who scores partially proficient, proficient, or advanced proficient 
on a state language arts test may also show some congruency in the level of 
achievement in his/her well-aligned school work and classroom assessments. As 
teachers become better calibrated, they will be able to answer questions such as: What 
level of proficiency does this student show on class assessments? Is the difficulty level 
of the class work comparable to the state exam? What can I do to help this student 
meet the state standards? Is my program meeting the standards? 

PREDICTING OUTCOMES 



Teachers can reflect upon their level of calibration accuracy by attempting to predict 
student results on state assessments. Teachers should be aware, however, that 100% 
agreement should not be expected between a student's performance on well-calibrated 
classroom tests and on the state assessments, based on many factors of test design. 
To begin the prediction process, the teacher uses a list of the students taking the test. 
Beside each name, the teacher enters a predicted score level. When the state 
assessment scores arrive, the teacher can compute the level of accuracy as shown 
below. 

Name Prediction Score 
Allan Proficient Adv. Proficient 
Ann Proficient Proficient 
Tamika Adv. Proficient Proficient 
Bronson Partial Proficient Partial Proficient 



The list above shows a 50% level of success in the predictions made. 

The teacher making the predictions returns to each student's work and compares the 
successful predictions with the unsuccessful ones to gain a better idea of how the 
assessment performances reflect the aligned student work. Student work associated 
with actual test scores can form the basis for subsequent calibration discussions. 
Student work connected to state assessment score levels can also function as scoring 
examples that students refer to when judging their own work. 
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FINAL THOUGHTS 



There is a distinct difference between traditional notions of test preparation and aligning 
and calibrating instruction and assessments with the content, cognition, difficulty, and 
format of state assessment instruments, specifications, and standards. As educators, 
we are trying to link the classroom activities to the standards and skills set by the state. 
The aim is to ensure that teachers understand, and calibrate their classrooms with 
respect to, the entire process and do not simply focus on how to answer specific types 
of test questions. The questions will change, but the underlying skills and concepts will 
not. One must be careful not to wallow in the mire of test prep. 

Delineation, alignment, and calibration are academic endeavors that demand unending 
commitment. Do not expect to accomplish alignment or calibration at an in-service day, 
or even during the course of a school year. The administration must provide the time 
and resources to conduct frequent calibration meetings to examine such things as 
classroom work and student assessment samples. Beware, it is easy to fall out of 
alignment and calibration and into test prep. 
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